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NON-LINEAR WAVEFRONT CODING SYSTEMS AND METHODS 

RELATED APPLICATIONS 
[0001] This application claims priority to U.S. provisional application serial 
number 60/449,944, filed 25 February 2003 and hereby incorporated by reference. 

BACKGROUND 

[0002] Traditional optical imaging system design has focused on the 
combination of certain optics with an image detector. The optics may include, for 
example, lenses that focus and magnify images. The image detector may be a non-linear 
detector (e.g. film) or a linear detector (e.g., CCD array) that detects the optical radiation 
to form an image. The detected image may then undergo traditional image processing 
techniques - such as edge sharpening and color filter array (CFA) color interpolation - to 
improve certain aspects of the image. Thus, traditional optics in imaging systems are 
designed to form images at the detector that are sharp and clear over a range of field 
angles, illumination wavelengths, temperatures, and focus positions. 

[0003] Traditional imaging systems, however, suffer from a number of 
setbacks. First, such systems typically have a relatively low depth of field, making it 
difficult to accurately and fully image real objects. The depth of field refers to the range of 
distances over which an optical imaging system can produce an in- focus image. Since the 
range of the depth of field is inherently limited, there are problems associated with 
imaging an object wherein its dimensions exceed the depth of field, as some portion of the 
image will be out of focus and blurry. To overcome this low depth of field, additional 
optical elements may be employed, for example an optical power absorbing apodizer or a 
stopped down aperture. However, overcoming low depth of field in these ways 
significantly reduces the optical energy at the image plane and decreases image resolution. 
The decrease of optical energy and increase in image noise may be at least partially 
remedied by adding significant illumination to the traditional imaging system; but this 
adds weight, complexity and expense to the design. 

[0004] Misfocus-related aberrations, and control of such, present other 
problems in traditional imaging systems. Misfocus-related aberrations include, for 
example, chromatic aberration, curvature of field, spherical aberration, astigmatism, and 
temperature or pressure related misfocus; such aberrations cause image blurring and loss 
of resolution, and can make image colors appear shifted or otherwise incorrect. 
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Traditional imaging systems remedy such aberrations by adding lens elements that 
increase cost, weight and overall lens complexity. 

[0005] Consequently, traditional imaging systems must make a trade-off 
between (a) good images that generally require complex optical designs and (b) lower 
quality images that reduce size, weight, and/or the number of optical elements in the 
optical design. 

[0006] Extended depth of field incoherent optical imaging systems 
(sometimes denoted as WAVEFRONT CODED® imaging systems) have been developed 
as an improvement over traditional imaging systems. These systems extend the depth of 
field (or depth of focus) properties in the imaging system by encoding the wavefront 
("wavefront coding") as it is processed through the imaging system. U.S. Patent No. 
5,748,371, issued to Cathey, Jr. et al., provides certain useful background on wavefront 
coding and is incorporated herein by reference. 

[0007] One principle underlying wavefront coding is that an optical system, 
under incoherent illumination, forms a linear system with respect to intensity. As a result, 
wavefront coding is used with linear digital detectors (e.g., CCD array, CMOS arrays, etc.) 
since digital detectors can have a linear response to the intensity of optical radiation. 
Photographic film and many other detectors do not, however, demonstrate such a linear 
response. Even digital detectors, especially CMOS detectors, can be intentionally 
designed to exhibit a non-linear response. One reason to design such non-linearities is to 
extend the dynamic range of the detector; for example, the circuit around each pixel may 
be configured to produce a response that is logarithmic instead of linear. Another reason 
to design such non-linearities is to cause any portion of the image below a threshold of the 
detector not to be imaged. 

SUMMARY OF THE INVENTION 

[0008] In one embodiment, a non-linear optical system images an object. 
Optics focus and encode a wavefront from the object to an intermediate image such that an 
optical transfer function of the optical system, at the intermediate image, is more invariant 
to focus-related aberrations as compared to an intermediate image formed by the optics 
without encoding. A non-linear detector detects the intermediate image. A linearization 
processor (e.g., implementing a linearization process) electronically captures the 
intermediate image. 
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[0009] In another embodiment, a method reduces aberrations in a wavefront 
imaged by an optical system having a non-linear detector. A wavefront of electromagnetic 
radiation from an object imaged to the non-linear detector is encoded. Data from the non- 
linear detector is digitally converted to form a digital representation of the image captured 
by the non-linear detector. The detected image is linearized to form a linearized image. 
The linearized image is filtered to reverse effects of wavefront coding to form a final 
image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0010] FIG. 1 depicts a non-linear wavefront coded imaging system. 

[0011] FIG. 2A shows ray intercept diagrams for a traditional optical imaging 

system. 

[0012] FIG. 2B shows ray intercept diagrams for the optical imaging system 
of FIG. 1. 

[0013] FIG. 3 shows a graph of an exposure curve for a non-linear detector. 

[0014] FIG. 4 illustrates a linearization process for use with image capture by 
a non-linear detector. 

[0015] FIG. 5 shows a graph of a non-linear exposure curve for negative film 
and linear exposure curve. 

[0016] FIG. 6A illustrates a series of modulation transfer functions 
corresponding to the series of point spread functions for a wavefront imaged on an analog 
detector without linearization. 

[0017] FIG. 6B shows a series of modulation transfer functions corresponding 
to the series of point spread functions for a wavefront imaged on a non-linear detector with 
linearization. 

DETAILED DESCRIPTION OF THE INVENTION 

[0018] A non-linear imaging system 100 that extends depth of field is shown 
generally in FIG. 1. System 100 includes (a) optics 102 that may, for example, take the 
form of an optical phase mask, (b) optical elements (e.g., lenses and/or mirrors) 104, (c) a 
non-linear detector 106, and (d) a post processor 108. Optics 102 and elements 104 
cooperate to image electromagnetic radiation 52 from object 50 to an intermediate image 
103, which is then captured by non-linear detector 106. Electromagnetic radiation 52 may 
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be in one or more of the visible spectrum, infrared spectrum, ultraviolet spectrum, radio 
wave spectrum, or another part of the electromagnetic spectrum. Optics 102 "encode" the 
optical wavefront within system 100, as described below, to spatially blur points of imaged 
object 50 about intermediate image 103 (e.g., along optical axis 105). 

[0019] Intermediate image 103 does not, therefore, occupy a single focal point 
as in traditional imaging systems. An exemplary ray path-based explanation of the 
difference between traditional imaging systems and imaging system 100 is illustrated in 
FIGS. 2A and 2B. In particular, FIG. 2A shows optical rays 50 from an ideal traditional 
lens focusing parallel light. Optical rays 50 converge to a focal point 52 at the optical axis 
54, with the best focus image plane at 50mm on the horizontal axis 56 (the vertical axis 58 
represents distance from the on-axis position). The ray density of the traditional imaging 
system is thus maximally sensitive to movement of the image plane or misfocus. FIG. 2B, 
on the other hand, illustrates optical rays 60 from imaging system 1 00 (in an exemplary 
embodiment), with optics 102 providing a cubic phase surface. As shown, optical rays 60 
no longer travel towards a point of best focus, but instead travel so that the distribution of 
rays 60 is substantially insensitive to position of the image plane. Intermediate image 1 03 
of an object point is therefore not a point image, but is instead a specialized blur. 

[0020] A linearization process 110 linearizes data from detector 106. By way 
of example, if non-linear detector 106 is photographic film, linearization process 110 may 
include a digital scanner than scans the photographic image of intermediate image 103. In 
another example, if non-linear detector 106 is a non-linear digital detector (e.g., a CMOS 
array with a non-linear circuit, or detector elements with individual, non-linear responses), 
then linearization process 110 may include digital processing to linearize digital data from 
detector 106. In one embodiment, data from detector 106 is linearized by linearization 
process 1 10 to form linearized image 1 12 by utilizing a priori knowledge of the exposure 
curve of detector 106 (for example the gamma curve of photographic film as detector 106), 
and/or by measuring the exposure curve. 

[0021] In one embodiment, linearized image 1 12 is a linear representation of 
intermediate image 1 03 which would have been captured by a linear detector in place of 
non-linear detector 106. In the embodiment, post-processor 108 then processes or filters 
linearized image 1 12 by removing wavefront coding effects of mask 102 to form a final 
image 114 with increased depth of field and/or reduced focus-related aberration(s). Final 
image 1 14 is a sharp (e.g., in-focus) image as compared to intermediate image 103. 
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[0022] In one embodiment, optics 102 include a phase profile (e.g., circularly 
symmetric or non-circularly symmetric profile (e.g., cubic)) that modifies the optical 
transfer function of imaging system 100 by affecting a phase of light transmitted by the 
phase profile. Optics 102 may be positioned at or near a principal plane of system 100; at 
or near an image of the principal plane; at or near an aperture stop of system 100; at or 
near an image of the aperture stop, or at or near a lens (e.g., a lens 104) of system 100, so 
that non-linear detector 1 06 presents the only substantial non-linearity in system 100. 
Non-linear detector 1 06 may be made of any type of non-linear media, such as 
photographic film or specially designed digital detectors that exhibit a non-linear response 
based on exposure. In yet another aspect, the non-linearity is such that the portion of the 
image below a threshold of the non-linear detector is not recorded. 

[0023] As noted above, linearization process 110 may include digital scanning 
of non-linear detector 1 06 with a conventional digital scanner. An exemplary digital 
scanner configured for scanning photographic film as non-linear detector 106 may be a 
Nikon Super Coolscan 4000 film scanner at 4000 dots per inch, with a bit depth of 8-bits 
and with sixteen times averaging to reduce CCD array noise. An exemplary photographic 
film as non-linear detector 106 may be Kodak T-Max 100 black-and-white negative film, 
which may be developed with a Kodak T-Max developer in a 1 :3 dilution for seven 
minutes at 72°F, prior to being digitally scanned, such as illustrated in FIG. 3. 

[0024] More particularly, FIG. 3 shows a graph of exposure versus density in 
a typical negative film, illustrating properties of detector 1 06 when detector 1 06 is this 
particular film. Underexposed regions of the film fall to the left of curve 30 (i.e., within 
non-linear regions 34A) and overexposed regions of the film fall to the right of curve 30 
(i.e., within non-linear regions 34B). There is a linear region 32 to curve 30; however, the 
dynamic range of a typical image captured on the film will often extend outside of linear 
region 32 and into non-linear areas 34. Typically, negative films are overexposed rather 
than risking underexposure, so it is more likely that images will be overexposed in non- 
linear regions 34B. The underexposed region may fall below an intensity threshold so that 
no image is recorded in that underexposed region. 

[0025] In one embodiment, linearization process 110 for a particular non- 
linear detector 106 begins with estimating the highest density in intermediate image 103 
captured by detector 106. Alternatively, the highest density in intermediate image 103 is 
measured with a densitometer, such that density estimation is unnecessary. Then, using a 
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part of the non-linear exposure curve of non-linear detector 106 that ranges from the 
lowest region of the curve (e.g., underexposed region 34A of curve 30, FIG. 3) to the 
estimated or measured highest density (e.g., region 34B, FIG. 3), a look up table is 
constructed by solving for integer values of the non-linear exposure curve to map its 
values to generate linearized image 112. 

[0026] Linearized image 112 may be generated by the linearization process 
200 shown in FIG. 4. At step 202, a non-linear exposure curve is generated by capturing a 
series of images of uniform intensity across an image plane of system 100, to create a 
uniform density on film. The captured images vary by exposure time and are digitally 
scanned and utilized within linearization process 110. With the non-linear exposure curve, 
at step 204, the density values are converted to grey values. Then, at each sample image 
captured on the non-linear exposure curve, the grey value is used as an address in a lookup 
table, and the value of a linear curve at that sample image is used as the data at that address 
in the lookup table, at step 206. Using this lookup table, the captured images are scanned 
through, and the grey value of each pixel that originally resided on the non-linear exposure 
curve is swapped with the linearized value entered at the corresponding address in the , 
lookup table, at step 208. 

[0027] Linearization process 200 is described for black-and-white imaging 
with non-linear detector 106. Process 200 may also be applicable for color imaging, for 
example with color film as non-linear detector 106, by performing the steps of process 200 
separately for each of the red, green and blue channels in the detected intermediate image 
103. In this way, post processor 108 then combines the separate channels to form final 
image 114. 

[0028] FIG. 5 shows an exemplary graph of negative film exposure versus 
density of the captured images and plots for the non-linear exposure curve 300 and the 
linear exposure curve 302. These are exposure curves for Kodak T-Max 100 film, 
developed in a Kodak T-Max developer for seven minutes at 72°F. As exposure increases 
for a particular image, the linearization process brings about noise gain in the image; 
however, this is offset by improved contrast. Specifically, contrast, which is related to the 
slope of the exposure curve, decreases upwardly on non-linear exposure curve 300 into the 
overexposed region; but the contrast on linear exposure curve 302 is constant. Therefore, 
the higher the image intensity on non-linear exposure curve 300, the greater the noise gain 
realized; however increased contrast offsets the noise gain. The increased contrast and 
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increased depth of focus of imaging system 100 thus provide benefit when imaging with 
non-linear detector 106. 

[0029] Post processor 108 performs filtering on linearized image 1 12 to form 
final image 1 14, by taking into account the spatial blur applied by optics 102 while 
preserving the increase in depth of field and/or the reduction of focus-related aberrations. 
In one exemplary arrangement of post processor 108 as a filter, a Fourier-domain equation 
is used to calculate the filter: 

W(v 9 QH*jv 9 Q 

where F(?, ?) is the filter, Wis the ideal diffraction-limited OTF, and His the input data 

is the complex conjugate of H). H is an OTF given by an experimentally determined 
PSF. The above equation is computed separately at each frequency pair (?, ?) to build the 
filter. The bandwidth of the filter is determined by the bandwidth of the ideal OTF, W. 
The ideal OTF represents on-axis linear imaging at a best focus position, s is the Weiner 
parameter. 

[0030] The difference in PSFs among different exposure levels on non-linear 
detector 106 presents a concern, as these levels are considered noise. To estimate the 
value of the Weiner parameter, the variance of the difference between the PSF used to 
determine the filter (one of the set of various exposure levels, or an average of several 
exposure levels) and the PSFs of different exposure levels is calculated. A reliable 
estimate for s is the largest of the variances divided by four. Another metric to determine 
s is the noise gain of the filter, given by the square root of the sum of squared filter 
coefficients, given that the filter coefficients sum to one. The larger the value of s , the 
smaller the noise gain. Noise gains near two are generally the best results when acting on 
the set of PSFs of different exposure levels. 

[0031] In an experimental set-up using Kodak T-Max 100 film as non-linear 
detector 106 (developed in a Kodak T-Max developer for seven minutes at 72°F), a series 
of point spread functions (PSFs) were developed, each successive PSF image having 
double the exposure time of the previous PSF. These non-linearized PSFs were taken 
directly from a film scanner. Then, the PSFs were linearized and filtered by a filter 
generated by the average of the four middle exposed non-linearized PSFs. The resulting, 
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linearized were more clearly alike, as compared to the non-linearized PSFs, thus providing 
more consistent response over varying exposure levels. Moreover, energy within the 
linearized PSFs appeared more concentrated in the center of each image, and there was 
also less noise within the images. 

[0032] FIG. 6A shows normalized MTFs corresponding to each non- 
linearized PSF described above in connection with the experimental set-up. MTFs are the 
modulus of the corresponding OTFs, and are plotted as magnitude versus normalized 
frequency values. The MTFs of FIG. 6A show the effects of non-linearities by the 
separation between MTFs for different exposure values. If the optical imaging system 
were linear, then the normalized MTFs would substantially lie on top of one another, as 
the only significant difference between the MTFs would be a scaling factor, which is 
removed when the MTFs are normalized. The shape of each MTF in FIG. 6A also varies 
across the normalized frequency values for each exposure, illustrating the non-linearities. 

[0033] FIG. 6B shows the normalized MTFs corresponding to each linearized 
PSF from the experimental set-up. These MTFs are closer together across the normalized 
frequency values for each exposure. Because of improved this linearization, removal of 
effects of wavefront coding, by post processor 108, is more easily achieved. 

[0034] Since certain changes may be made in the above methods and systems 
without departing from the scope hereof, it is intended that all matter contained in the 
above description or shown in the accompanying drawing be interpreted as illustrative and 
not in a limiting sense. It is also to be understood that the following claims are to cover 
certain generic and specific features described herein. 
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